Skip to content

Conversation

@gabry-lab
Copy link

@gabry-lab gabry-lab commented Jan 16, 2026

Closes #1302

@github-actions
Copy link
Contributor

Thanks for your contribution!

This PR doesn't have a linked issue. All PRs must reference an existing issue.

Please:

  1. Open an issue describing the bug/feature (if one doesn't exist)
  2. Add Fixes #<number> or Closes #<number> to this PR description

See CONTRIBUTING.md for details.

@github-actions
Copy link
Contributor

The following comment was made by an LLM, it may be inaccurate:

Potential Duplicate Found:

Why they might be related: Both PRs address dynamic API key management. PR #8198 adds apiKeyCommand for dynamic API key retrieval, while PR #8817 adds apiKeyHelper and apiKeyRefreshInterval options. These appear to be alternative or overlapping approaches to solving the same problem of managing API keys dynamically. You should review PR #8198 to ensure there's no conflicting implementation or to see if they should be consolidated.

@Haoping-Xiao
Copy link

@thdxr please review this PR. this is a super important feature for users using litellm proxy with rotating api key expiratiion

@dottharun
Copy link

The getLanguage function which does the getSDK call also caches the model with model.providerId and model.id as key,

so for long running sessions, this will provide stale sdk without calling the apiHelper again

export async function getLanguage(model: Model): Promise<LanguageModelV2> {
const s = await state()
const key = `${model.providerID}/${model.id}`
if (s.models.has(key)) return s.models.get(key)!

@gabry-lab
Copy link
Author

The getLanguage function which does the getSDK call also caches the model with model.providerId and model.id as key,

so for long running sessions, this will provide stale sdk without calling the apiHelper again

export async function getLanguage(model: Model): Promise<LanguageModelV2> {
const s = await state()
const key = `${model.providerID}/${model.id}`
if (s.models.has(key)) return s.models.get(key)!

What do you think how we should resolve it?

@schuettc
Copy link

This would work well for our use case - we have a credential helper that issues short-lived tokens (55-minute validity for tokens), so the apiKeyRefreshInterval option is exactly what we need.

Our setup:

  • Enterprise deployment using a custom API gateway
  • Credential helper script that fetches JWT tokens
  • Tokens expire after 60 minutes, so we'd set apiKeyRefreshInterval to ~55 minutes

Happy to help with this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Request to support Dynamic API keys through an apiKeyHelper

5 participants